Stereo in the Presence of Specular Reeection

The problem of accurate depth estimation using stereo in the presence of specular reection is addressed. Specular re ection is viewpoint dependent and can cause large intensity di erences at corresponding points. Hence, mismatches can result causing signi cant depth errors. Current stereo algorithms largely ignore specular re ection which is a fundamental re ection phenomenon from surfaces, both smooth and rough. We analyzed the physics of specular re ection and the geometry of stereopsis which led us to an interesting relationship between stereo vergence, surface roughness, and the likelihood of a correct match. Given the lower bound on surface roughness, an optimal binocular stereo con guration can be determined which maximizes precision in depth estimation despite specular re ection. However, surface roughness is di cult to estimate in unstructured environments. Therefore, multiple view con gurations independent of surface roughness are determined such that at each scene point visible to all sensors, at least one stereo pair provides a correct depth estimate. We have developed a simple algorithm to reconstruct depth from the multiple view images. Experiments with real surfaces conrm the viability of our approach. A key feature of this approach is that we do not seek to eliminate or avoid specular re ection, but rather minimize its e ect on stereo matching. Categories: Stereo, Physics-based vision i